<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"
                "http://www.w3.org/TR/REC-html40/loose.dtd">
<html>
<head>
  <title>Description of demgpard</title>
  <meta name="keywords" content="demgpard">
  <meta name="description" content="DEMGPARD Demonstrate ARD using a Gaussian Process.">
  <meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
  <meta name="generator" content="m2html &copy; 2003 Guillaume Flandin">
  <meta name="robots" content="index, follow">
  <link type="text/css" rel="stylesheet" href="../../m2html.css">
</head>
<body>
<a name="_top"></a>
<div><a href="../../menu.html">Home</a> &gt;  <a href="#">ReBEL-0.2.7</a> &gt; <a href="#">netlab</a> &gt; demgpard.m</div>

<!--<table width="100%"><tr><td align="left"><a href="../../menu.html"><img alt="<" border="0" src="../../left.png">&nbsp;Master index</a></td>
<td align="right"><a href="menu.html">Index for .\ReBEL-0.2.7\netlab&nbsp;<img alt=">" border="0" src="../../right.png"></a></td></tr></table>-->

<h1>demgpard
</h1>

<h2><a name="_name"></a>PURPOSE <a href="#_top"><img alt="^" border="0" src="../../up.png"></a></h2>
<div class="box"><strong>DEMGPARD Demonstrate ARD using a Gaussian Process.</strong></div>

<h2><a name="_synopsis"></a>SYNOPSIS <a href="#_top"><img alt="^" border="0" src="../../up.png"></a></h2>
<div class="box"><strong>This is a script file. </strong></div>

<h2><a name="_description"></a>DESCRIPTION <a href="#_top"><img alt="^" border="0" src="../../up.png"></a></h2>
<div class="fragment"><pre class="comment">DEMGPARD Demonstrate ARD using a Gaussian Process.

    Description
    The data consists of three input variables X1, X2 and X3, and one
    target variable  T. The  target data is generated by computing
    SIN(2*PI*X1) and adding Gaussian  noise, x2 is a copy of x1 with a
    higher level of added noise, and x3 is sampled randomly from a
    Gaussian distribution. A Gaussian Process, is trained by optimising
    the hyperparameters  using the scaled conjugate gradient algorithm.
    The final values of the hyperparameters show that the model
    successfully identifies the importance of each input.

    See also
    <a href="demgp.html" class="code" title="">DEMGP</a>, <a href="gp.html" class="code" title="function net = gp(nin, covar_fn, prior)">GP</a>, <a href="gperr.html" class="code" title="function [e, edata, eprior] = gperr(net, x, t)">GPERR</a>, <a href="gpfwd.html" class="code" title="function [y, sigsq] = gpfwd(net, x, cninv)">GPFWD</a>, <a href="gpgrad.html" class="code" title="function g = gpgrad(net, x, t)">GPGRAD</a>, <a href="gpinit.html" class="code" title="function net = gpinit(net, tr_in, tr_targets, prior)">GPINIT</a>, <a href="scg.html" class="code" title="function [x, options, flog, pointlog, scalelog] = scg(f, x, options, gradf, varargin)">SCG</a></pre></div>

<!-- crossreference -->
<h2><a name="_cross"></a>CROSS-REFERENCE INFORMATION <a href="#_top"><img alt="^" border="0" src="../../up.png"></a></h2>
This function calls:
<ul style="list-style-image:url(../../matlabicon.gif)">
<li><a href="gp.html" class="code" title="function net = gp(nin, covar_fn, prior)">gp</a>	GP	Create a Gaussian Process.</li><li><a href="gpcovar.html" class="code" title="function [cov, covf] = gpcovar(net, x)">gpcovar</a>	GPCOVAR Calculate the covariance for a Gaussian Process.</li><li><a href="gpfwd.html" class="code" title="function [y, sigsq] = gpfwd(net, x, cninv)">gpfwd</a>	GPFWD	Forward propagation through Gaussian Process.</li><li><a href="gpinit.html" class="code" title="function net = gpinit(net, tr_in, tr_targets, prior)">gpinit</a>	GPINIT	Initialise Gaussian Process model.</li><li><a href="netopt.html" class="code" title="function [net, options, varargout] = netopt(net, options, x, t, alg);">netopt</a>	NETOPT	Optimize the weights in a network model.</li></ul>
This function is called by:
<ul style="list-style-image:url(../../matlabicon.gif)">
<li><a href="demnlab.html" class="code" title="function demnlab(action);">demnlab</a>	DEMNLAB A front-end Graphical User Interface to the demos</li></ul>
<!-- crossreference -->


<h2><a name="_source"></a>SOURCE CODE <a href="#_top"><img alt="^" border="0" src="../../up.png"></a></h2>
<div class="fragment"><pre>0001 <span class="comment">%DEMGPARD Demonstrate ARD using a Gaussian Process.</span>
0002 <span class="comment">%</span>
0003 <span class="comment">%    Description</span>
0004 <span class="comment">%    The data consists of three input variables X1, X2 and X3, and one</span>
0005 <span class="comment">%    target variable  T. The  target data is generated by computing</span>
0006 <span class="comment">%    SIN(2*PI*X1) and adding Gaussian  noise, x2 is a copy of x1 with a</span>
0007 <span class="comment">%    higher level of added noise, and x3 is sampled randomly from a</span>
0008 <span class="comment">%    Gaussian distribution. A Gaussian Process, is trained by optimising</span>
0009 <span class="comment">%    the hyperparameters  using the scaled conjugate gradient algorithm.</span>
0010 <span class="comment">%    The final values of the hyperparameters show that the model</span>
0011 <span class="comment">%    successfully identifies the importance of each input.</span>
0012 <span class="comment">%</span>
0013 <span class="comment">%    See also</span>
0014 <span class="comment">%    DEMGP, GP, GPERR, GPFWD, GPGRAD, GPINIT, SCG</span>
0015 <span class="comment">%</span>
0016 
0017 <span class="comment">%    Copyright (c) Ian T Nabney (1996-2001)</span>
0018 
0019 clc;
0020 randn(<span class="string">'state'</span>, 1729);
0021 rand(<span class="string">'state'</span>, 1729);
0022 disp(<span class="string">'This demonstration illustrates the technique of automatic relevance'</span>)
0023 disp(<span class="string">'determination (ARD) using a Gaussian Process.'</span>)
0024 disp(<span class="string">' '</span>);
0025 disp(<span class="string">'First, we set up a synthetic data set involving three input variables:'</span>)
0026 disp(<span class="string">'x1 is sampled uniformly from the range (0,1) and has a low level of'</span>)
0027 disp(<span class="string">'added Gaussian noise, x2 is a copy of x1 with a higher level of added'</span>)
0028 disp(<span class="string">'noise, and x3 is sampled randomly from a Gaussian distribution. The'</span>)
0029 disp(<span class="string">'single target variable is given by t = sin(2*pi*x1) with additive'</span>)
0030 disp(<span class="string">'Gaussian noise. Thus x1 is very relevant for determining the target'</span>)
0031 disp(<span class="string">'value, x2 is of some relevance, while x3 should in principle be'</span>)
0032 disp(<span class="string">'irrelevant.'</span>)
0033 disp(<span class="string">' '</span>);
0034 disp(<span class="string">'Press any key to see a plot of t against x1.'</span>)
0035 pause;
0036 
0037 ndata = 100;
0038 x1 = rand(ndata, 1);
0039 x2 = x1 + 0.05*randn(ndata, 1);
0040 x3 = 0.5 + 0.5*randn(ndata, 1);
0041 x = [x1, x2, x3];
0042 t = sin(2*pi*x1) + 0.1*randn(ndata, 1);
0043 
0044 <span class="comment">% Plot the data and the original function.</span>
0045 h = figure;
0046 plotvals = linspace(0, 1, 200)';
0047 plot(x1, t, <span class="string">'ob'</span>)
0048 hold on
0049 xlabel(<span class="string">'Input x1'</span>)
0050 ylabel(<span class="string">'Target'</span>)
0051 axis([0 1 -1.5 1.5])
0052 [fx, fy] = fplot(<span class="string">'sin(2*pi*x)'</span>, [0 1]);
0053 plot(fx, fy, <span class="string">'-g'</span>, <span class="string">'LineWidth'</span>, 2);
0054 legend(<span class="string">'data'</span>, <span class="string">'function'</span>);
0055 
0056 disp(<span class="string">' '</span>);
0057 disp(<span class="string">'Press any key to continue'</span>)
0058 pause; clc;
0059 
0060 disp(<span class="string">'The Gaussian Process has a separate hyperparameter for each input.'</span>)
0061 disp(<span class="string">'The hyperparameters are trained by error minimisation using the scaled.'</span>)
0062 disp(<span class="string">'conjugate gradient optimiser.'</span>)
0063 disp(<span class="string">' '</span>);
0064 disp(<span class="string">'Press any key to create and train the model.'</span>)
0065 disp(<span class="string">' '</span>);
0066 pause;
0067 
0068 net = <a href="gp.html" class="code" title="function net = gp(nin, covar_fn, prior)">gp</a>(3, <span class="string">'sqexp'</span>);
0069 <span class="comment">% Initialise the parameters.</span>
0070 prior.pr_mean = 0;
0071 prior.pr_var = 0.1;
0072 net = <a href="gpinit.html" class="code" title="function net = gpinit(net, tr_in, tr_targets, prior)">gpinit</a>(net, x, t, prior);
0073 
0074 <span class="comment">% Now train to find the hyperparameters.</span>
0075 options = foptions;
0076 options(1) = 1;
0077 options(14) = 30;
0078 
0079 [net, options] = <a href="netopt.html" class="code" title="function [net, options, varargout] = netopt(net, options, x, t, alg);">netopt</a>(net, options, x, t, <span class="string">'scg'</span>);
0080 
0081 rel = exp(net.inweights);
0082 
0083 fprintf(1, <span class="keyword">...</span>
0084   <span class="string">'\nFinal hyperparameters:\n\n  bias:\t\t%10.6f\n  noise:\t%10.6f\n'</span>, <span class="keyword">...</span>
0085   exp(net.bias), exp(net.noise));
0086 fprintf(1, <span class="string">'  Vertical scale: %8.6f\n'</span>, exp(net.fpar(1)));
0087 fprintf(1, <span class="string">'  Input 1:\t%10.6f\n  Input 2:\t%10.6f\n'</span>, <span class="keyword">...</span>
0088   rel(1), rel(2));
0089 fprintf(1, <span class="string">'  Input 3:\t%10.6f\n\n'</span>, rel(3));
0090 disp(<span class="string">' '</span>);
0091 disp(<span class="string">'We see that the inverse lengthscale associated with'</span>)
0092 disp(<span class="string">'input x1 is large, that of x2 has an intermediate value and the variance'</span>)
0093 disp(<span class="string">'of weights associated with x3 is small.'</span>)
0094 disp(<span class="string">' '</span>);
0095 disp(<span class="string">'This implies that the Gaussian Process is giving greatest emphasis'</span>)
0096 disp(<span class="string">'to x1 and least emphasis to x3, with intermediate emphasis on'</span>)
0097 disp(<span class="string">'x2 in the covariance function.'</span>)
0098 disp(<span class="string">' '</span>)
0099 disp(<span class="string">'Since the target t is statistically independent of x3 we might'</span>)
0100 disp(<span class="string">'expect the weights associated with this input would go to'</span>)
0101 disp(<span class="string">'zero. However, for any finite data set there may be some chance'</span>)
0102 disp(<span class="string">'correlation between x3 and t, and so the corresponding hyperparameter remains'</span>)
0103 disp(<span class="string">'finite.'</span>)
0104 disp(<span class="string">'Press any key to continue.'</span>)
0105 pause
0106 
0107 disp(<span class="string">'Finally, we plot the output of the Gaussian Process along the line'</span>)
0108 disp(<span class="string">'x1 = x2 = x3, together with the true underlying function.'</span>)
0109 xt = linspace(0, 1, 50);
0110 xtest = [xt', xt', xt'];
0111 
0112 cn = <a href="gpcovar.html" class="code" title="function [cov, covf] = gpcovar(net, x)">gpcovar</a>(net, x);
0113 cninv = inv(cn);
0114 [ytest, sigsq] = <a href="gpfwd.html" class="code" title="function [y, sigsq] = gpfwd(net, x, cninv)">gpfwd</a>(net, xtest, cninv);
0115 sig = sqrt(sigsq);
0116 
0117 figure(h); hold on;
0118 plot(xt, ytest, <span class="string">'-k'</span>);
0119 plot(xt, ytest+(2*sig), <span class="string">'-b'</span>, xt, ytest-(2*sig), <span class="string">'-b'</span>);
0120 axis([0 1 -1.5 1.5]);
0121 fplot(<span class="string">'sin(2*pi*x)'</span>, [0 1], <span class="string">'--m'</span>);
0122 
0123 disp(<span class="string">' '</span>);
0124 disp(<span class="string">'Press any key to end.'</span>)
0125 pause; clc; close(h); clear all
0126</pre></div>
<hr><address>Generated on Tue 26-Sep-2006 10:36:21 by <strong><a href="http://www.artefact.tk/software/matlab/m2html/">m2html</a></strong> &copy; 2003</address>
</body>
</html>